Single Hidden Layer Neural Network Research Articles

Flat minima generalize for low-rank matrix recovery

Abstract Empirical evidence suggests that for a variety of overparameterized nonlinear models, most notably in neural network training, the growth of the loss around a minimizer strongly impacts its performance. Flat minima—those around which the loss grows slowly—appear to generalize well. This work takes a step towards understanding this phenomenon by focusing on the simplest class of overparameterized nonlinear models: those arising in low-rank matrix recovery. We analyse overparameterized matrix and bilinear sensing, robust principal component analysis, covariance matrix estimation and single hidden layer neural networks with quadratic activation functions. In all cases, we show that flat minima, measured by the trace of the Hessian, exactly recover the ground truth under standard statistical assumptions. For matrix completion, we establish weak recovery, although empirical evidence suggests exact recovery holds here as well. We complete the paper with synthetic experiments that illustrate our findings.

Analysis Of Exchange Rate Forecast Model of US Dollar Against RMB Based on BP Neural Network

Judging from the economic development situation in recent years, the uncertainty of exchange rate fluctuation will be affected by the exchange rate, not only for domestic and foreign trade but also for the cost of enterprises and individuals. Therefore, in view of this crucial problem, in this study, BP(Back Propagation) synthetic neural network is drilled by the opening price of the day in order to predict the closing price in the next day. The method is to divide 5000 groups of data into 4000 training sets and 1000 test sets. Single hidden layer neural network is adopted, and the number of nodes is determined to be 6 and 15 through the experiment of the influence of the quantity of nodes on model performance. The simulation test show that the root mean square errors in the prediction set and the verification set are very small, which are 0.0004 and 0.0006 respectively, so the neural network has good accuracy in forecasting the closing price of the next day, and some opinions on this model are also given.

Neural Network Methods Based on Efficient Optimization Algorithms for Solving Impulsive Differential Equations

In view of the fact that impulsive differential equations have the discreteness due to the impulse phenomenon, this paper proposes a single hidden layer neural network method based extreme learning machine and a physicsinformed neural network method combined with learning rate attenuation strategy to solve linear impulsive differential equations and nonlinear impulsive differential equations respectively. For the linear impulsive differential equations, firstly, the interval is segmented according to the impulse points, and a single hidden layer neural network model is constructed, the weight parameters of hidden layer are randomly set, the optimal output parameters and solution of the first segment are obtained by the extreme learning machine algorithm, then we calculate the initial value of the second segment according to the jumping equation, and the remaining segments are solved in turn in the same way. Although the single hidden layer neural network method proposed can solve linear equations with high accuracy, it is not suitable for solving nonlinear equations. Therefore, we propose the physics-informed neural network combined with learning rate attenuation strategy to solve the nonlinear impulsive differential equations, then the Adam algorithm and L-BFGS algorithm are combined to find the optimal approximate solution of each segment. Numerical examples show that the single hidden layer neural network method with Legendre polynomials as the activation function and the physics-informed neural network method combined with learning rate attenuation strategy can solve linear and nonlinear impulsive differential equations with higher accuracy.

Association between short birth spacing and child malnutrition in Bangladesh: a propensity score matching approach

ObjectivesThis study aimed to explore the effects of short birth spacing (SBS), which is defined as a period of less than 33 months between two successive births, on multiple concurrent...

A new meshless method of solving the distributed-order time-fractional mobile-immobile equations

A new method is introduced by combining neural network’s huge expressive power with technologies of the minimum residual approximate solutions (MRASs). In spatial direction, a single hidden layer neural network makes the construction of bases functions easier. The cost of calculations becomes acceptable in high dimensions space using the technologies of the MRASs. And this paper establish approximation and convergence order theories based on neural network.

Research and application of safety monitoring technology of distribution automation based on SOM neural network

Abstract For automatic situation monitoring in power distribution safety monitoring, data features are mainly extracted by single hidden layer neural network, which makes the standard error of monitoring results larger. Therefore, the research and application of power distribution automation safety monitoring technology based on SOM neural network are proposed. Preprocess the power grid, collect distribution operation data, set multi-dimensional monitoring nodes according to the collected data, build a distribution operation status monitoring model and analyze the data fusion technology of distribution automation safety monitoring according to the model. On this basis, the distribution automation safety monitoring system is defined, the output node correction weight and the reverse output node correction weight are calculated, the SOM neural network identification model is constructed and the research and application of the distribution automation safety monitoring technology are completed under the action of the gravitational function between individuals within the target time. The experimental results show that the change curve of the number of vulnerabilities and the actual number of false positives is consistent, the number of vulnerabilities is small and the monitoring results are more accurate; The state of the safety monitoring equipment of distribution automation is normal. After applying the method in this paper, the change curve is consistent with the actual value, and the delay time is less than 15 ms.

A Simplified Current Feature Extraction and Deployment Method for DC Series Arc Fault Detection

DC series arc fault (DSAF) is one of the main causes of low-voltage DC (LVDC) distribution system fire accident. Traditional methods use time-frequency domain features to make judgments directly or use machine learning to further classify them. In this paper, a method is proposed to simplify the feature extraction process while ensuring accuracy. Firstly, a fully automated process is used to establish DC arc datasets. A composite bandpass filter (CBF) is designed to extract the typical frequency segment of DC arc. Besides, an arc detection neural network (ArcNN) based on temporal convolution network (TCN) is proposed to extract current waveform features. Principal component analysis (PCA) is used to process these features to reduce correlation. Finally, a single hidden layer neural network (SHLNN) is used as classifier. The database is collected from different scenarios and working conditions. By measuring the DC raw current, the arc fault detector (AFD) can achieve a test set accuracy of 99.88% at a sampling rate of 250 kHz. The model is also deployed on Jetson Nano with an average real-time detection of 0.15 s/sample.

Nonlinear Temperature Control of Additive Friction Stir Deposition Evaluated on an Echo State Network

Abstract Additive friction stir deposition is a recent innovation in additive manufacturing allowing the deposition of metallic alloys onto a metallic deposit bed, creating a purely mechanical metallic bond. The deposition can be done in a layer-by-layer manner, and the purely mechanical process eliminates the need for high energy consumption and can be deposited at a much higher rate than beam-based welding. The mechanical nature of the process allows the bonding of dissimilar alloys and a reduction in size of the heat-affected zone. The additive friction stir deposition process is difficult to model and existing literature has focused on numerical analysis, which is not amenable to online closed-loop control. In this work, a form of reservoir computing called an echo state network is used to model the additive friction stir deposition process from online process data, and validation is performed on a reserved dataset. Subsequently, a model-free controller using Lyapunov-derived combination of the robust integral of the sign error, and a single hidden layer neural network design is developed to control the additive friction stir deposition process. Control efficacy is given by way of a Lyapunov analysis which shows the system is globally exponentially stable, and simulation results with the echo state networks. Stability proof shows that under one assumption, the controller can be extrapolated to the real system. The mean squared error of the tracking result using the controller and echo state network simulation is 2.05 °C.

Deterministic Weight Modification-based Extreme Learning Machine for Stock Price Prediction

Background: The prediction of the stock price is considered to be one of the most fascinating and important research and patent topics in the financial sector. Aims: Making more accurate predictions is a difficult and significant task because the financial industry supports investors and the national economy. Objectives: The DWM is used to adjust the connection weights and biases to enhance prediction precision and convergence rate. DWM was proposed as a method to reduce system error by changing the weights of various levels. The methods for predictable changes in weight were provided together with the computational difficulty. Methods: An extreme learning machine (ELM) is a fast-learning method for training a singlehidden layer neural network (SLFN). However, the model's learning process is ineffective or incomplete due to the randomly chosen weights and biases of the input's hidden layers. Hence, this article presents a deterministic weight modification (DWM) based ELM called DWM-ELM for predicting the stock price. Results: The calculated results showed that DWM-ELM had the best predictive performance, with RMSE (root mean square error) of 0.0096, MAE (mean absolute error) of 0.0563, 0.0428, MAPE (mean absolute percentage error) of 1.7045, and DS (Directional Symmetry) of 89.34. conclusion: The experimental results showed that, in comparison to other well-known prediction algorithms, the suggested DWM+ELM prediction model offers better prediction performance. Conclusion: The experimental results showed that, in comparison to other well-known prediction algorithms, the suggested DWM+ELM prediction model offers better prediction performance.

Approximation error of single hidden layer neural networks with fixed weights

Neural networks with finitely many fixed weights have the universal approximation property under certain conditions on compact subsets of the d-dimensional Euclidean space, where approximation process is considered. Such conditions were delineated in our paper [26]. But for many compact sets it is impossible to approximate multivariate functions with arbitrary precision and the question on estimation or efficient computation of approximation error arises. This paper provides an explicit formula for the approximation error of single hidden layer neural networks with two fixed weights.

Feature Weighting for Parkinson's Identification using Single Hidden Layer Neural Network

The diagnosis of Parkinson has become easier with the existence of machine learning. It includes using existing features from the biometric dataset generated by the person to identify whether he has Parkinson or not. The features differ in their discrimination capability and they suffer from redundancy. Hence, researchers have recommended using feature selection for Parkinson's identification. The feature selection aims at finding the most important and relevant features to produce an efficient and effective model. In this article, we present entropy-based Parkinson classification. The goal is to select only 50% of the most relevant features for Parkinson prediction. Two variants of neural networks are used for evaluation, the first one is a feed-forward Extreme Learning Machine ELM and the second one is Fast Learning Machine FLN. Also, the K-Nearest Neighbor KNN algorithm is used for evaluation. The results show the superiority of ELM and FLN when the model of feature selection is used with an accuracy of 80% compared with only 78% when the model is not used.

An optimization-based supervised learning algorithm for PXRD phase fraction estimation

In powder diffraction data analysis, phase identification is the process of determining the crystalline phases in a sample using its characteristic Bragg peaks. For multiphasic spectra, we must also determine the relative weight fraction of each phase in the sample. Machine learning algorithms (e.g., Artificial Neural Networks) have been applied to perform such difficult tasks in powder diffraction analysis, but typically require a significant number of training samples for acceptable performance. We have developed an approach that performs well even with a small number of training samples. We apply a fixed-point iteration algorithm on the labelled training samples to estimate monophasic spectra. Then, given an unknown sample spectrum, we again use a fixed-point iteration algorithm to determine the weighted combination of monophase spectra that best approximates the unknown sample spectrum. These weights are the desired phase fractions for the sample. Our approach gave the best accuracy and lowest mean absolute error when compared with several machine learning algorithms, including a single hidden-layer neural network.

Speech emotion recognition based on syllable-level feature extraction

Speech emotion recognition systems have high computational requirements for deep learning models and low generalizability mainly because of the poor reliability of emotional measurements across multiple corpora. To solve these problems, we present a speech emotion recognition system based on a reductionist approach of decomposing and analyzing syllable-level features. Mel-spectrogram of an audio stream is decomposed into syllable-level components, which are then analyzed to extract statistical features. The proposed method uses formant attention, noise-gate filtering, and rolling normalization contexts to decrease contextual differences and increase focus the attention on the structure of a formant. A set of syllable-level formant features is extracted and fed into a single hidden layer neural network that makes predictions for each syllable as opposed to the conventional approach of using a sophisticated deep learner to make sentence-wide predictions. The syllable level predictions help to lower the aggregated error in utterance level cross-corpus predictions. The experiments on IEMOCAP (IE), MSP-Improv (MI), and RAVDESS (RA) databases show that the method archives better than the state-of-the-art cross-corpus unweighted accuracy of 47.6% for IE to MI and 56.2% for MI to IE.

An artificial electric field algorithm-based neuro-fuzzy predictor for estimation of compressive strength of concrete structures: A machine learning approach

Machine learning based forecasting are found better to manual and statistical methods in estimating compressive strength of concrete structures. However, there is need of exploring an effective, automated and accurate predictor for this domain. This article proposes an artificial electric field algorithm-based neuro-fuzzy network (AEFA+NFN) for prediction of compressive strength of concrete structures. A single hidden layer neural network (SHNN) is used as the base model and its inputs are fuzzified using Gaussian triangular membership function with a degree of membership to different classes. The optimal number of input data, hidden neurons, bias and weights for hidden layer are decided by AEFA. The model is evaluated on samples from a publicly available dataset with curing ages at 3, 7, 14, and 28 days. Considering four sample series, the AEFA+NFN produced an average MAPE of 0.092073 and ARV of 0.139731 which are better compared to others. The experimental outcomes and analysis are in favor of the AEFA+NFN based forecasting.

Cathode Position Detection in a Transferred Arc Plasma Using Artificial Neural Network

In a transferred arc plasma system, the position of the cathode is difficult to detect during the smelting process as it remains inside the cylindrical anode. Real-time and accurate cathode position detection leads to efficient smelting operation with optimal use of electrical energy. In this article, a machine learning technique is proposed to accurately detect the position of the cathode in a direct current (DC) transferred arc plasma system. The measured voltage signal sampled at 20 kHz is processed using a tunable Q-factor wavelet transform (TQWT) followed by statistical features extraction and a machine learning algorithm to provide accurate cathode position information. Two different machine learning algorithms are used in this work, namely, single hidden layer neural network (SHLNN) and single-layer extreme learning machine (SELM). The output of these machine learning algorithms provides accurate position information and is also compared to the traditional voltage-related position information. The experimental signal of a 30-kW DC plasma system and cathode position detection results is shown.

Low-rank kernel approximation of Lyapunov functions using neural networks

<p style='text-indent:20px;'>We study the use of single hidden layer neural networks for the approximation of Lyapunov functions in autonomous ordinary differential equations. In particular, we focus on the connection between this approach and that of the meshless collocation method using reproducing kernel Hilbert spaces. It is shown that under certain conditions, an optimised neural network is functionally equivalent to the RKHS generalised interpolant solution corresponding to a kernel function that is implicitly defined by the neural network. We demonstrate convergence of the neural network approximation using several numerical examples, and compare with approximations obtained by the meshless collocation method. Finally, motivated by our theoretical and numerical findings, we propose a new iterative algorithm for the approximation of Lyapunov functions using single hidden layer neural networks.</p>

Model Predictive Control Coupled with Artificial Intelligence for Eddy Current Dynamometers

The recent studies on Artificial Intelligence (AI) accompanied by enhanced computing capabilities supports increasing attention into traditional control methods coupled with AI learning methods in an attempt to bringing adaptiveness and fast responding features. The Model Predictive Control (MPC) technique is a widely used, safe and reliable control method based on constraints. On the other hand, the Eddy Current dynamometers are highly nonlinear braking systems whose performance parameters are related to many processes related variables. This study is based on an adaptive model predictive control that utilizes selected AI methods. The presented approach presents an updated the mathematical model of an Eddy Current Dynamometer based on experimentally obtained system operational data. Finally, the comparison of AI methods and related learning performances based on the assessment technique of mean absolute percentage error (MAPE) issues are discussed. The results indicate that Single Hidden Layer Neural Network (SHLNN), General Regression Neural Network (GRNN), Radial Basis Network (RBNN), Neuro Fuzzy Network (ANFIS) coupled MPC have quite satisfying performances. The presented results indicate that, amongst them, GRNN appears to provide the best performance.

Research on Coal Dust Wettability Identification Based on GA-BP Model.

Aiming at the problems of the influencing factors of coal mine dust wettability not being clear and the identification process being complicated, this study proposed a coal mine dust wettability identification method based on a back propagation (BP) neural network optimized by a genetic algorithm (GA). Firstly, 13 parameters of the physical and chemical properties of coal dust, which affect the wettability of coal dust, were determined, and on this basis, the initial weight and threshold of the BP neural network were optimized by combining the parallelism and robustness of the genetic algorithm, etc., and an adaptive GA−BP model, which could reasonably identify the wettability of coal dust was constructed. The extreme learning machine (ELM) algorithm is a single hidden layer neural network, and the training speed is faster than traditional neural networks. The particle swarm optimization (PSO) algorithm optimizes the weight and threshold of the ELM, so PSO−ELM could also realize the identification of coal dust wettability. The results showed that by comparing the four different models, the accuracy of coal dust wettability identification was ranked as GA−BP > PSO−ELM > ELM > BP. When the maximum iteration times and population size of the PSO algorithm and the GA algorithm were the same, the running time of the different models was also different, and the time consumption was ranked as ELM < BP < PSO−ELM < GA−BP. The GA−BP model had the highest discrimination accuracy for coal mine dust wettability with an accuracy of 96.6%. This study enriched the theory and method of coal mine dust wettability identification and has important significance for the efficient prevention and control of coal mine dust as well as occupational safety and health development.

Machine learning models for 180-day mortality prediction of patients with advanced cancer using patient-reported symptom data

PurposeThe objective of the current study was to develop and test the performances of different ML algorithms which were trained using patient-reported symptom severity data to predict mortality within 180 days for patients with advanced cancer.MethodsWe randomly selected 630 of 689 patients with advanced cancer at our institution who completed symptom PRO measures as part of routine care between 2009 and 2020. Using clinical, demographic, and PRO data, we trained and tested four ML algorithms: generalized regression with elastic net regularization (GLM), extreme gradient boosting (XGBoost) trees, support vector machines (SVM), and a single hidden layer neural network (NNET). We assessed the performance of algorithms individually as well as part of an unweighted voting ensemble on the hold-out testing sample. Performance was assessed using area under the receiver-operating characteristic curve (AUROC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).ResultsThe starting cohort of 630 patients was randomly partitioned into training (n = 504) and testing (n = 126) samples. Of the four ML models, the XGBoost algorithm demonstrated the best performance for 180-day mortality prediction in testing data (AUROC = 0.69, sensitivity = 0.68, specificity = 0.62, PPV = 0.66, NPV = 0.64). Ensemble of all algorithms performed worst (AUROC = 0.65, sensitivity = 0.65, specificity = 0.62, PPV = 0.65, NPV = 0.62). Of individual PRO symptoms, shortness of breath emerged as the variable of highest impact on the XGBoost 180-mortality prediction (1-AUROC = 0.30).ConclusionOur findings support ML models driven by patient-reported symptom severity as accurate predictors of short-term mortality in patients with advanced cancer, highlighting the opportunity to integrate these models prospectively into future studies of goal-concordant care.

Application of machine learning models to investigate the performance of stainless steel type 904 with agricultural waste

In this study, machine learning algorithm was used to model measurements of corrosion rates of stainless steel Type 904 in 0.5 M H2SO4 media as a function of exposure time, and temperature, when corrosion inhibitors are added in different dosage. Hyper-parameter optimization of the adaptive neuro-fuzzy inference system (ANFIS) and artificial neural network (ANN) was carried out to select the best model that accurately predicts the corrosion rate. A range of 1–10 neurons in a single hidden layer neural network with varying prominent training algorithms and activation functions was tested. The FCM-clustered ANFIS model tested a range of 2–20 cluster numbers. The models’ performance was evaluated using correlation determination (R2), root mean square error (RMSE), mean absolute percentage error (MAPE), and mean absolute deviation (MAD). The experiments show that the FCM-clustered ANFIS model with 20 clusters outperformed all other models with MAPE, MAD, RMSE, and R2 values of 1.9222, 0.5543, 0.7317, and 0.9308 at the testing phase respectively, and 0.5457, 0.3943, 2.9258 and 0.9993 at the training phase respectively. The entire FCM-ANFIS model produced the optimum prediction with a 96.40% level of robustness and accuracy. Therefore, ANFIS and ANN can be used to predict corrosion inhibition using agricultural waste in acidic media with great degree of success. These models are reliable, cheap, fast, efficient, and effective tools for corrosion inhibition prediction.